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EXAMINER'S ANSWER 



This is in response to the appeal brief filed on June 10, 2005 
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( 1) Real Party in Interest 

A statement identifying the real party in interest is contained in the brief. 

(2) Related Appeals and Interferences 

A statement identifying the related appeals and interferences which will directly 
affect or be directly affected by or have a bearing on the decision in the pending appeal 
is contained in the brief. 

(3) Status of Claims 

The statement of the status of the claims contained in the brief is correct. 

(4) Status of Amendments A fter Final 

The appellant's statement of the status of amendments after final rejection 
contained in the brief is correct. 

(5) Summary of claimed subject matters 

The summary of claimed subject matter contained in the brief is correct even 
though the title of the section is not correct. 

(6) Grounds of rejection to be reviewed on appeal 

The appellant's statement of grounds of rejection to be reviewed on appeal in the 
brief is incorrect. The correct number of Bergman should be US 6,564,263. It is 
appeared a typo for this number. 

(7) Claims Appealed 
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The copy of the appealed claims contained in the Appendix of Claims to the brief is 
correct. 

(8) Prior Art of Record 

6,564,263 Bergman etal. 05-2003 

6,512,850 Yaung 01-2003 



(9) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 

Claim Rejections - 35 USC § 102 

1 . Claims 1-2. 4-7, 9-1 1 , 13-18, 28-33 and are rejected under 35 U.S.C. 102(e) as 
being anticipated by US 6564263 to Bergman et al.. 

As to claim 1, Bergman discloses a content-based multimedia retrieval system, 
comprising: 

a first color quantizer which extracts a color histogram of query multimedia data 
(col. 13 lines 45-col. 14 line 2, a first quantizer can be myhist:histogram(interger)[512]); 

a second quantizer which extracts a color histogram of query multimedia data 
(col. 13 lines 45-col. 14 line 2, a second quantization can be 
rgbhist:histogram(interger)[512] or HVShist:histogram(real[166] ,see in col. 13 line 7); 
and 

a histogram converter (fig. 19, Fi(Q), i=1, 2, and 3, note that the transform F 
converts histogram form rgb color space to a new color space) which converts the 
color histogram of one of the extracted query multimedia data and the multimedia data 
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to be retrieved Into a histogram having a color space and color quantization method 
(color quantization method quantizes the colors to 512 bins or 512 levels, or may be 
166 levels in HVS color space, col. 13 line 6) of the other of the extracted query 
multimedia data and the multimedia data to be retrieved, (fig. 19, col. 13 line 16-col. 14 
line 2). 

As to claim 2, Bergman further discloses the multimedia data are image data or 
video data (fig. 9 and fig. 16). 

As to claim 4, Bergman further discloses description means for describing color 
spaces and color quantization methods (e.g., rgb512, note that rgb describes RGB color 
space and 512 describes 512 bin histogram) (fig. 19, col. 13 lines 45-col. 14 line 2) 

As to claim 5, Bergman further discloses converting the histogram of the query 
multimedia data so as to be corresponding to color space and color quantization 
method of the multimedia data to be retrieved (fig. 19, col. 13 line 18-col. 14 line 40). 

As to claims 6-7, the claims are the corresponding method claims to claims 1-3, 
respectively. The discussion are addressed with regard to claims 1-3. 

As to claim 9, Bergman further discloses: 

judging whether the color histogram of query multimedia data is extracted before 
(fig. 19, col. 13 line 18-col. 14 line 40; note that finding whether the user generated 
histogram Q is the judgment); 

reading a color histogram value extracted before (user generated before) and the 
multimedia data to be retrieved (archives) and identifying based color space and 
quantization method (fig. 19, col. 13 line 18-col. 14 line 40, note that it is inherent); and 

converting the color histograms Into the color histograms of the same color space 
and color quantization method when the histograms are not same (fig. 19, col. 13 line 
18-col. 14 line 40). 
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As to claims 10-11, Bergman further discloses extracting the histogram when 
there is no user created histogram of the inputted query data (fig. 19, col. 13 line 18-col. 
14 line 40), and the converting process is performed by referencing the color space 
description info and quantization info of the multimedia data to be retrieved and query 
data (fig. 19, col. 13 line 18-col. 14 line 40), 

As to claim 17, Bergman discloses a content-based multimedia retrieval method, 
comprising: 

comparing the color spaces and color quantization methods of the query 
multimedia data an multimedia data to be retrieved (fig. 19, col. 13 line 24-col. 14 line 2, 
see remark); 

converting the color histogram of the query multimedia data when the color 
space and color quanization method of the query multimedia data and the color space 
space and the color quantization method of the multimedia to be retrieved are different 
(col. 13 line 24-col. 14 line 2, see also remark); and 

calculating a similarity between the converted query multimedia and multimedia 
data to be retrieved, and performing a retrieval in accordance with the calculated 
similarity (col. 13 line 24-col. 14 line 2, note that the fundamental similarity functions 
calculate the distance of different descriptors). 

As to claim 18, Berman further discloses converting the color histogram (myhist) 
of query multimedia to the color histogram of multimedia data to be retrieved (fig. 19). 

As to claims 13-16, 28-33, the discussions are addressed with regard to claims 
1-11 and 17-18. 
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Claim Rejections - 35 USC § 103 

2. Claim 12 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Bergman in view of US 6512850 to Yaung. 

As to claim 12, Bergman does not explicitly mention comparing the similarity with 
a certain threshold value which is well known in the art. 

Yaung, in an analogous environment, discloses the well known concept (13, line 
46-col. 14 line 67). 

It would have been obvious to one having ordinary skill in the art at the time the 
invention was made to use the scheme of Yaung in the system of Bergman to compare 
the similarity of histogram colors with a threshold in order to accurately identify the 
image to be retrieved. 

(10) Response to Argument 

3. (A) The following discussion relates to the rejection of claims 1 -2, 4-7, 9-11,1 3- 
18, 28-33 and are rejected under 35 U.S.C. 102(e) as being anticipated by US 
6564263 to Bergman et al.. 

1 . Appellants' argument -— Appellant argues for claims 1 , 2, and 5 that 1 ) 
"Bergman does not discloses that search engine 1902 performs histogram conversion 
based also on color quantization ." but only based on color space (page 8 paragraph 2); 
2) Bergnian does not teach the histogram conversion with a color space having color 
quantization (R: five levels, G: five levels, B: five levels) to another color space having 
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color quantization (H: seven levels, S: seven levels : V and seven levels), (page 9, 
paragraph 2). 

Examiner's response — The Examiner respectfully disagree with Appellant. 
First, the claimed language does not have the limitations as argued converting 
"histograms based on color quantization methods". Claimed language is "a histogram 
converter which converts the color histogram of one of the extracted query multimedia 
data and the multimedia data to be retrieved into a histogram having a color space and 
color quantization method of the another of the extracted query multimedia data and the 
multimedia data to be retrieved." (emphasis by the Examiner). Second, there is no 
support in the specification for the argument of converting histogram based on 
quantization. Thus, it is difficult for the Examiner response the argument made by the 
Appellant. The Examiner assumes that the Appellant's argument directed to the claimed 
limitation. Second, Appellant's argument on lack of the color quantization in Bergman 
reveals his misunderstanding of basic principle of color image processing and his 
mischaracterization the Bergman, which is one of the reason we have this Appeal Brief 
now. As well known to one ordinary in the art, the color histogram such as RGB 
histogram, has 3 single color histograms with each having certain bins or levels 
(sometime called quantization levels, as in Bergman 512 levels) as one coordinate and 
number of pixels falling the levels as another coordinate. Moreover, anyone ordinary 
skilled in the art would understand that when one uses color histogram, one has to use 
color quantization to form bins or levels which is one coordinate of the histogram and 
another coordinate is the number of pixels falling in the bin or the number of histogram 
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having the levels. Here, Bergman's RGBhist: histogram (integer) [512] is short written of 
(R: 512 bins/levels, G; 512 bins/leves, B: 512 bins/levels), which uses integer 
quantization or HVShist:histogram (real) [166] is short written of (H: 166 bins/levels; V: 
166 bins/levels; S: 166 bins/levels, which uses real number quantization. In addition, 
Bergman defines myhist in column 13 line 50 as myhist: histogram (interger)[512), 
which is 512 integer quantization level histogram to character a color space. Bergman 
also expressly teaches the histogram conversion in column 13 line 55, 
myhist=F(rgbhist), which means converts rgbhist defined in column 13 line 46 to myhist 
defined in column 13 line 50 by the transformation F. Thus, Bergman explicitly teaches 
the limitations in the claims 1 , 2, and 5 of "a histogram converter which converts the 

color histogram into a histogram having a color space and color quantization 

method...". Furthermore, in response to Appellant's argument that the references fail to 
show certain features of Appellant's invention, it is noted that the features upon which 
Appellant relies (i.e.. the histogram conversion with a color space having color 
quantization (R: five levels, G: five levels, B: five levels) to another color space having 
color quantization (H: seven levels, S: seven levels : V and seven levels) are not recited 
in the rejected claim(s). Although the claims are interpreted in light of the specification, 
limitations from the specification are not read into the claims. See In re Van Geuns, 988 
F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Finally, Bergman has done the same as 
that of the invention for describing a multimedia objects by content and facilitates 
content-based search, index, and retrieval (abstract) 
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Therefore, Appellant's argument is incorrect because of misunderstanding and 
mischaracterization of Bergman. 

2. Appellants' argument — Appellant also argues for claim 4 that Bergman does 
not teach color space descriptor 306 and color quantization descriptor (page 10, 
paragraph 1). 

Examiner's response — The Examiner respectfully disagree with Appellant. 
First, Bergman expressly mentions RGB color space and corresponding histogram as 
rgbihst:histogram (integer) [512] (col. 13 lines 44-47or HVS color space and 
corresponding histogram as HVShis:histogram (real)[166] (col. 13 lines 4-6). Moreover, 
Bergman teaches using a InforPyramid model to describe a multimedia data (fig. 16). 
This model includes several standard descriptors, "such as color, texture motion, etc. for 
example, 166 bin color histograms derived from HVS color space may preferably be 
defined as : HVShist:histogram(real)[166]". (col. 13, line 3-7). Finally, those are the 
descriptors for color space and corresponding histogram. 

Therefore, Appellant's argument is incorrect because of misunderstanding and 
mischaracterization. 

3. Appellants' argument — Appellant further argues for claims 6-7 and 1 1 that 
Examiner fails to teach that content-based multimedia retrieval which includes 
"converting a color histogram having a color space and color quantization method of the 
other input query multimedia data and multimedia data to be retrieved" and Bergman 's 
conversion does not " based on color space and color quantization" (page 10, paragraph 
2. emphasis added by the Examiner). 
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Examiner's response — The Examiner respectfully disagree with Appellant. 
First, correct claim language is "having" not "based on". Second, Bergman expressly 
teach that converting rgbhist to myhist (also see section A(1)). Furthermore, Bergman 
expressly teach using the conversion in the multimedia data retrieval (see abstract). 

Therefore, Appellant's argument is incorrect because of misunderstanding and 
mischaracterization. 

4. Appellants' argument — Appellant further argues for claim 9-1 0 that Bergman 
does not teach "judging whether there is a color histogram of the query image which is 
extracted before (S501 ). If yes, the method reads a color histogram value of the query 
image extracted before (S502), reads a color histogram value of a feature database 
about a retrieval object image (S504), determines whether the color histogram of the 
query image is different from the color histogram (color space and color quantization 
method) of the image to be retrieved (S505), converts the color histogram value of the 
query image (S506), and calculates similarity by comparing the color histogram of the 
query image with the color image to be retrieved (S507). The method then determines 
whether the similarity is large than a certain threshold value (S508), and if yes, outputs 
the image to be retrieved as a similar image (S509)." (page 11-12, paragraph 1). 

Examiner's response — The Examiner respectfully disagree with Appellant. 

First, what the Appellant argues above is not the same as the claimed language 
of claims 9 and 10 but in the specification. Although the claims are interpreted in light 
of the specification, limitations from the specification are not read into the claims. See 
In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Second, since 
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Appellant did not present a point by point argument for claims 9 and 10, it is difficult for 
the Examiner responses. However, Bergman teaches the all limitations of claims 9 and 
10 as presented in the final rejection. Finally, Bergman clearly addresses the problem 
the different color space and histogram between the query multimedia data and stored 
multimedia data. (fig. 19, col. 13 line 57-col. 14 line 2). In addition, Bergman teaches 
that "[l]n Oder for the search engine to query the multiple archives, given a single query 
color histogram Q (preferably generated by a user 1901 ), the search engine 1902 must 
transform that query histogram Q into the appropriate histogram color space o f the 
particular archives 1903, 1904, 1905 (i.e., F1(Q), F2(Q), and F3(Q) respectively and 
those transformations are done by the transform functions F (col. 13, lines 39-55). 
Content-based searching across multiple archives requires transformation of the query 
histogram Q to be compatible with the specific content descriptions in each archives." 
(fig. 19, col. 13 line 60-col. 14 line 2). Needless to say to start transformation, one need 
to know whether or not the archived multimedia data is compatible with the query 
multimedia data, thus, the comparison has to be inherent. 
Therefore, Appellant's argument is incorrect. 

5. Appellants' argument — Appellant still further argues for the claims 13-16 that 

1 ) Bergman does not discloses "comparing the extracted color space and color 
quantization method of the query image with the color space and color quantization 
method of the multimedia data to be retrieved" (emphasis added)" (page 12, paragraph 

2) ; and 2) Bergman does not disclose or suggest "converting the histogram pf one of 
the extracted query multimedia data and the multimedia data to be retrieved into color 
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histogram having same color space and color quantization method as of the extracted 
query multimedia data and the multimedia data to be retrieved." (page 12, paragraph 
3). 

Examiner's response — The Examiner respectfully disagree with Appellant. 
Regarding 1 ), in the instant case, Bergman clearly addresses the problem the different 
color space and histogram between the query multimedia data and stored multimedia 
data. (fig. 19, col. 13 line 57-col. 14 line 2). In addition, Bergman teaches that "[l]n oder 
for the search engine to query the multiple archives, given a single query color 
histogram Q (preferably generated by a user 1901), the search engine 1902 must 
transform that query histogram Q into the appropriate histogram color space o f the 
particular archives 1903, 1904, 1905 (i.e., F1(Q), F2(Q), and F3(Q) respectively and 
those transformations are done by the transform functions F" (col. 13, lines 39-55). 
Content-based searching across multiple archives requires transformation of the query 
histogram Q to be compatible with the specific content descriptions in each archives, 
(fig. 19, col. 13 line 60-col. 14, line 2). Needless to say to start transformation, one need 
to know whether or not the archived multimedia data is compatible with the query 
multimedia data, thus, the comparison has to be inherent operation. Furthermore, one 
skilled in the art would transform those color space and color histogram are not 
compatible with the archived multimedia data. Finally, the model includes also "[t]he 
fundamental description functions preferably comprises several classes, such as logic, 
similarity and transform functions, among others. Fundamental logic functions 
contemplated by the present invention may include, for example, "equal", "not equal", 
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"greater-than", "less-than", "and", "or" "mot" etc. as known by those skilled in the art." 
(col. 13 lines 24-29). Those functions clearly can be used for comparing different color 
descriptors for multimedia data, for example, using the function "equal" or "not equal" to 
two color space/histogram descriptors. 

Therefore, it is clear to those skilled in the art that Bergman uses the descriptors 
to describe color spaces and histograms of each of multimedia data in archives and the 
query multimedia data. Bergman further inherently use the fundamental logic function to 
decide those to be to be retrieved having different descriptors. If any difference found, 
the search engine will conduct "transformations of the query histogram Q to be 
compatible with the specific content descriptions in each archives." (col. 13 line 67-col. 
14 line 2). 

Regarding to 2), sections A(1 ) and A (5)(1 ) have addressed the argument. 
Therefore, Appellant's argument is incorrect. 

6. Appellants' argument — Appellant still argues for claims 17-18 that "Bergman 
does not disclose or suggest comparing color quantization methods performed for query 
color histogram Q and the color histograms formed for the Images stored in its archives. 
Also, the claimed conversion is also performed based on color quantization method, a 
conversion which is not disclosed or suggested in Bergman." (page 1 3, paragraph 2). 

Examiner's response — The Examiner respectfully disagree with Appellant. The 
response is addressed with regard to sections A(1 ) and A(5. 

Therefore, Appellant's argument is incorrect. 
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7 Appellants' argument — Appellant still further argues for claim 28 that 1 ) 
Bergman does not disclose or suggest a description means. As previously discussed, 
Betgman does not disclose that, when performing its color histogram transformations, 
search engine 1902 takes color quantization method into consideration." (page 13, 
paragraph 4); 2) Bergman does not teach that "a second color quantizer which extracts 
a color histogram of query multimedia data using a method which is same as the 
described color space and color quantization method" determined by the descriptor 
means." (page 14, paragraph 1). 

Examiner's response — The Examiner respectfully disagree with Appellant. 

Regarding 1 ), the response has been addressed with regard to the Section A(2). 

Regarding to 2), Bergman expressly mention that myhist that can be the second 
quantizer which uses 512 integer color quantization level that is the same as the rgbhist 
having also 512 color quantization levels, which all determined by the description 
means myhist: histogram (integer)[512] and rgbhis:histogram (integer)[512] (column 13 
lines 39-58). 

Therefore, Appellant's argument is incorrect. 

8. Appellants' arcument — Appellant still further argues for claims 29-30 that 
"Bergman does not disclose or suggest such a retrieval unit or database." 

Examiner's response — The Examiner respectfully disagree with Appellant. 

Bergman expressly mention the retrieval unit (search engine 1902 in fig. 19) to 
search several different archives (1903, 1904, 1905) that are inherently databases. In 
addition, Bergman further expressly teaches that the archives "may utilize a different 
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color histogram description, in order for tlie searcli engine to query the multiple archives 
given a query histogram Q (preferably generated by a user 1901), the search engine 
1902 must transform that query histogram Q into the appropriate histogram color 
spaces of the particular archives 1903, 1904, 1905 (i.e., F1(Q), F2(Q), and F3(Q), 
respectively)." (column 1 3 lines 56-66) 

Therefore, Appellant's argument is incorrect. 

9. Appellants' argument — Appellant still further argues for claim 31 that 
Bergnnan does not disclose or suggest color space descriptor and color quantization 
descriptor. 

Examiner's response — The Examiner respectfully disagree with Appellant. 
The response to the argument is addressed with regard to A(2). 
Therefore, Appellant's argument is incorrect. 

10. Appellants' argument — Appellant still further argues for claims 32-33 that 
Bergman does not disclose or suggest judging "histograms based on color quantization 
nor does it perform such a judgment in advance of performing a similarity calculation 
between the color histograms." 

Examiner's response — The Examiner respectfully disagree with Appellant. 

First, the claimed language does not have the limitations as argued judging 
"histograms based on color quantization". Claimed language is " judging whether a color 
histogram of query multimedia data corresponding to color space and quantization 
method of multimedia data to be retrieved is stored in advance before", which is not 
judging histogram based on guantization (emphasis by the Examiner). Second, there is 
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no support in the specification for the argument of judging histogram based on 
quantization. Thus, it is difficult for the Examiner response the argument made by the 
Appellant. The Examiner assumes that the Appellant's argument directed to the claimed 
limitations. Furthermore, Bergman expressly teaches that "each archives may utilize a 
different color histogram description. In order for the search engine to query the multiple 
archives given a query histogram Q (preferably generated by a user 1901), the search 
engine 1902 must transform that query histogram Q into the appropriate histogram color 
spaces of the particular archives 1903, 1904, 1905 (i.e., F1(Q), F2(Q), and F3(Q), 
respectively). Content-based searching across multiple archives requires 
transformations of the required histogram Q to be compatible with the specific content 
descriptions in each archives ." (column 13 line 56-column 14 line 2, emphasis added by 
the Examiner). Here, judging whether a color histogram Q is compatible to the 
multimedia data to be retrieved stored in advance is inherently a step before the 
transformation of the non-compatible color histogram Q because one need to know 
before whether or not the archived multimedia data is compatible with the query 
multimedia data Q, thus, the judging has to be Inherent operation. Finally, the emphasis 
quotation of Bergman illustrates that before one does the content-based searching (that 
is the similarity comparison of the content), one must transform (convert) the query 
histogram to be compatible with those in the archives. 

The response to the argument is also addressed with regard to A(6). 

Therefore, claims 1-2, 4-7, 9-11, 13-18 and 32-33 are property rejected under 35 
use §102. 
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(B). The following discussion relates to the rejection of claim 12 under 35 
U.S.C. 103(a) as being unpatentable over Bergman in view of Yaung. 

1 . Appellants' argument — Appellant argues that Yeung did not disclose the 
deficiency of Bergman, and thus claim 12 should be allowable. 

Examiner's response — The Examiner respectfully disagree with Appellant. 

First, the limitations regarding converting histogram are addressed with regard to 
A(1). Second, Yeung is operated in the same endeavor. Finally, Yeung is cited to show 
that the feature of comparing a similarity with certain threshold is well known in the art. 

It would have been obvious to one having ordinary skill in the art at the time the 
invention was made to use the scheme of Yaung in the system of Bergman to compare 
the similarity of histogram colors with a threshold in order to accurately identify the 
image to be retrieved. 

Therefore, claim 12 is properly rejected under 35 USC §103. 



For the above reasons, it is believed that the rejections should be 
sustained. 



Respectfully submitted 
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